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ABSTRACT 

This paper outlines the conceptiial framework 
underpinning the development of competency rating scales for adult 
literacy. It provides a rationale for a project that was conducted in 
Australia during International Literacy Year (1990) to develop 
appropriate ratings scales to assess adults' literacy and numeracy 
skills. The paper «!xamines the construct of literacy, defining 
pragmatic, cultural, functional, survival, or marginal adult literacy 
and suggesting that there can be many types and levels of literacy. 
Types of literacy identified include reading, writing, numeracy, 
document processing, and comMned skills. Three levels of literacy 
are proposed as useful: basic (minimum levels for gaining access to a 
culture); required (the skills necessary for a given social context, 
which may chaiige over time); and ah improvement level of literacy 
where an individua** lS able to take control of his or her life. Since 
these levels and ty^ iS of literacy exist, there is r basis for 
developing a set of rating scales for them, according to the paper. 
The paper provides several suggested methods of developing competency 
rating scales and profiles, included the preferred process used in 
the Australian project in which the starting point was the behaviors 
exhibited by pr»rsons in establishing literacy. (31 references) 
(KC) 
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This paper outlines rtw amceptual framework uniteiimuufig the development of competoicy 
rating scales for adult literacy. The paper examines the construct of literacy and ouUines a 
pxoceas for identifying indicaton of literacy developiront The use of these indicators in 
fbnning a set of c om p et ency rating scales is then outlined. 

Literacy: 

The Utoatuie provides us with many di£ferent definiticms of literacy and many diffmii 
contests in which the wimi lifie.acy is used. Two tilings are dear. First, it is not a neutral 
term aad second there are no yodse ddinitionsof the temu Utentty seems always to have 
onotional values attactod to it Wten we Ascribe sos^cme as literate or illitenue, our 
desciiptiim of that person carries widi it a lax^ amount of omstionai and value laden 
bagga^ Th» term is like the words liberty, justice or baf^nsess in that all have values 
associated with ttem. Thoe will ^obably not be a generally agreed definition of literacy 
and there will probably not be a goierally agreed way of meaauing literacy. Hence there 
is likely to be no final resdution of the issue of who is literate arMi who is not or even if this 
is even a sensible focus of discussion. 

Literacy programs are imponanL There are basic educaticm progiams in schools, colleges, 
tertiary institutions, in fimher education estabUshmoits and in tte wmkplace. There are 
programs workers, students, parents, and fw social groups. Tl» existeia of these 
courses indicates that we are concerned as a soci^y about a general lack of literacy, whatever 

we mean by the torn. 

We describe people as high or low in literacy or advanced in liter^. The notion of levels 
of literacy is enur^g. The term literate now seems to describe a person who is above a 
minimum levd of son» quality rather than an vgpes or adva^ed level. Literacy programs 
are designed to help individuals reach a minimal level of skill that can be used to describe 
an individual as literate. 

We have pragmatic, cultural, semi, conventional, funcdonal, survival or marginal and 
fiuKtional adult literacy. These are mostly terms which are associated with an individual's 
ability to peribrm some fiu^on rather than just to know som^hing. Function literacy has 
connotations of being inactical, usefU, and sodally relevant Cultural and functiimal literacy 
also carry a strong suggestion of the ccmtoct dependency of what, in terms of competence 
level, literacy actually means. But, in coo^doing models of linguistic competence, 
describing a structure and tieing this too narrowly to ^edfic situatiims ("the why, the when 
and the where") creates a poformanoe rather dian a competence model (Hickman, in 
Nystrand, 1986,27). 

The idea of 'functional literacy* suggests that there is a non functional literacy. Survival 
literacy suggests that there is a mm nuvival literacy. The use of the term "marginal literacy" 
suggests that there is a margin in which a perscm can e^tist outside the mainstream of literacy 
in society. Each of the terms suggests diat there is a pdnt at which a poson transforms 
from one classification to the other. That is, an individual can move from bcmg illiterate 
to literate; from being n<m function to functional; from non survival to survival or from the 
margins to the mainstream. What are the criteria for these transformations to take place and 
how can we recognise when the individual has changed status? 
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Wmzky (1990) aiggests that literacy xeqmres a set of ddUs for processing reacting and 
writmg rather than a stt of social skills that enable a perscm to cape or in facL to deceive and 
avoid the impaa of litenKy skills of others. ItoNsaremanyimiividualcasesof perscms who 
have becoii» very suocessfiil in business and society without the reading and witting skills. 
These iiutividuals usually enqtoy ottos to read for diem or enga^ in behaviour which 
avoids drcumsiances in which ihey must read. In many cases individuals with highly 
devdoped entrqneneuiial ddlls create a culture in whidi they are not requirBd to use these 
skills. In other cases, individuals may even go^ fax as deceit ami avmdance in order to 
escape (tetaction as a non leaito. Cases such as diese are common. 

In the recent Australian natimal study of adult literacy (Wicken, 1989) many assumptions 
about the tasks tnted were made. The seleoion of tasks was not based on any analysis of 
tte social, wcnkidaee or peiscmal literKy requirements of Au^ralian aduU pc^^ The 
tasks were ^ected by N(»di American researchers for the assessnwnt of North American 
youdi between the ages of 16 and 24. The tasks were divictod into the areas of reading, 
numeracy and document processing. Writing was not assessed but its importance was 
defiended in the assesanent of liten^y. 

The approach was explained as follows: 

It alhws for tite possibility <tf developing profiles noi only of a person's 
literacy preficiencies but also oftim U^raey remdrements of, say, particular 
occupations. Ami by using arui anafy^ d^ffkrem kimt of ttttm^ 
d^ffMng levels of com^adsy as the basts for assessment it allows for an 
assessment cfthe irftuence cfa text comruf^n itse^as a comributory factor 
to literacy d^fiadty and does not assume 0ua tntf 'problem' always resides 
widi die indtviduaL Above all it admowledges that to 'ftinaion' insodety is 
relative to individual needs and dm that diffierem levels and types of literacy 
need to be identified, so that profiles ofperformance can be constructed on the 
basis cf relevant and apprt^riate assessment tasks. OKHckot, 1989,4) 

This project focuses on identifying the types and levels of Utency to which Wickeit refers. 
It does not address the literacy requirements of particular occupations that she recommends. 
As explained in later sections of this paper, the pragect follows Wickert's advice by 
"...analysing different types of literacy tasks at differing levels of complexity as the basis of 
assessment..." 

Types of literacy 

The iype of litmcy skilli may be different for adults than those required for children. 
Adults opemt in a different sodal and cultural context. The school «\vironment is a 
nurturing one in which literacy is taught under the guise of reading and writing skills perhaps 
through phonics and perhaps through a whole language approach. The general competencies 
are the focus of instruction rather than specific tasks to complete. The tasks are used for 
assessment purposes or as the vehicle of instiucticn. Many individuals develop these general 
competencies. Many do not. Adult basic education and workplace basic education are 
predominantly concerned witlii those who do not succeed in gaining literacy skills in the 
school environment. 



This leaves the adult literacy field in a situaiiim where it is neoe?^ to identify the general 
comp^eodes needed to function effoctively, or the tasks which are needed in specific 
dxcumstances. Broad categories of dolls which might be potinent to the devdopn^nt of 
litoacy have been identified in the Uteranue. Kixsch and Jun^lut (1986) included four 
types of skills which are now gmrallyii^luded in definitions of literacy. These are reading, 
writing, numeracy and document procesang. Gemally these is no argumat about the 
inclusion of reading in tte definition of litoacy. However, there is no a|re«nent about what 
cCTiflitutes the baac or minimal level of competency in reading. 

Reading: 

The demands of the workplace environment for highly spca&Q tasks to be perfumed d^cax 
to be moving w<»kplace basic edtnatiim towards a task driven education. Specific tasks are 
learned rather than language competeroies. For those unable to read noterials firom & wide 
vaiiety of texts, the information they can use and the skills tiny spply, or can be trained to 
ai^y, have had to becon» very narrow aiKl very highly fbcussed. These pet^le med to be 
traiMd in processing infbrmation for highly specific tasks. Nwerthdess they need a basic 
set of infbrmation processing skills which have became defined as literacy skills. Many of 
these presume some very basic reading skills. 

Writing 

TlMic is no longer much argument about tte inclusicm of writing in the ctefiniticm of literacy. 
For some time, writing was coasktoed a practical task for whidi some imJividials had the 
"knack*. NWth the work of Graves in process writing, this area of liteacy has come to be 
seen as more than a practical task. Muchof the cognitive research rdated to writing focuses 
(m the componential theory of intelligei»e pn^wsed by Stonbog, which identifies the 
elements of performance to do widistxateeies and plans. For example, in the area of writing 
instnicticm. Flower has roistered a fundanmital shift from analysis of writtoi products to 
the focus on the interconmcting process of diinking, learning and writing (Hayes and 
Flower, 1986). In the sodo-cognitive approach taken by Nystrand (1986) the process of 
writing is one of negotiating und«standings and meanings betwcer. wiittt and reader. In 
describing a scale of language compmice, the identification of tiiese "metacognitive" 
elements, tiiose of the problem solving kind, is essential for a fiiU and accurate representation 
of what happens when adults write. 

Numeracy 

A third area which is gaining increasing attention is numeracy. There is growing consensus 
that at least basic competence in numeracy is required for functional literacy although there 
is again no agreement about how much numeracy is required nor for tiiat matter which basic 
numeracy skills are required. Numeracy beyond the basic operations and basic number 
systems is fer too complicated and qmalised io be included in a definition of basic literacy. 
Perhaps higher levels of numeracy and matiiematical sHlls become pait of the repertoire of 
performance of people who exhibit higher levels of Uteracy. Kirsch and Jung^lut (1986) 
confined their work to diose numeracy operations that are essential for the meaning of prim. 
These included basic addition and subtraction, comparisons, dates and time. 



Document Processing 



Hie fourth ar» which is now included in North American definitions of literacy is document 
processing. This has been a very difficuU area to deftiu in any way that enables an em{nricai 
analysis. Kirsch and Jungeblut (1986> study is the predominate study which introduced 
document processing as a compmnt of titoicy. Tteunrelatimis found b^ween this Utoacy 
scale ami those based on the reading and compretoision of extentted discourse (or prose), 
indicate that document processing may make an indepenttent comiibutifm to literacy 
devdopment. Alternatively, (tocument processing may be a »tb skill of reading or of 
numeracy or of both. So tte low correlations obtained may be an axte&ct of mixing 
lestriaed range com po ne nts of two scales and then trying to condate the scale to eadi. In 
other words the internal validity of the scates and the studies may be problematic and 
attempts to Inteiprst the ccarelatifma may be mitieading. Document prooesang may be an 
indwled component rather that a sqnxate conqxmit. It has oeen limited to the ability to 
deal with different types of document f irmats such as job ai^dications, TV time tables, 
advertisements, labels and so on. 

The Combined Skills: 

Not all of the skill areas play an equal role. They are mH all equally important. Venezky, 
Wagner and Ciliberti (1990) azgw that, while reading is inimary to any definito^ 
the other skills are secondary. Wridng as a imans of reccnding and communicating pre 
suf^sa reading. Otlnrwise it is a urechanical allying task. Reading pre supposes 
comprdhension otherwise it is recognition and or recitation. Numeracy and document 
processing are supplementary to reading BxtA have no role in literacy without it. 

A skilled reader who has little numeracy skills and who does not know much about the 
format of the documents will still be able to get agnificant armmnts of information from 
print. On the other hand persons spedficaUy trained to «>mpl^ particular forins and to pl^ 

both quantitative and verbal inforiiotion on those forms, may still stumble in an oivinmment 
which requires different skills in print aiui may not have the Utoacy skills to move firom one 
environment or task to another. In adult basic education the m^or concm is with those who 
do not read or read well. These intfividuals will need additional assistance in the fields of 
numeracy, writing and document processing (if the latter is a sqsarate and discrete skill). 
Rather than thinking about the specific tasks, we need to be thinking about the skills that 
oiable the tasks to be performed, ami to think about the psychological and cognitive skills 
9!nd competencies thai underpin those tasks. 

Levels of litera^^ 

Venezky, Wagner and CUiberti (1990) argues that there are two fundamental levels of 
literacy • basic and required literacy. Basic literacy has connotations of minimal levels that 
are necessary for self sustained development. It is the acquisition of a set of basic skills 
which enable the individual to gain access to a culture. This might t» called the access level. 

Required literacy encompasses the set of skills that are necessary for a given social context 
and may change over time, ^lioi the individual moves into a set social context, which 
require specific kinds of sldlls, such as in the work situation mors specific competounes 
may be needed. For example, one might want to become the secretary of a local sporting 
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dub, or to work in a paxticuiar job. These may incoipozate the language, numeracy and 
document pxocessing skills in varying degrees. Adjusmem mua be made if the skills are 
not already present, in order to develop to the required level of literacy. Howev», most 
of the researidi indicates that adults who have not aci|uired the language skills during their 
academic education at school, are unlikely to be able to be taught those generic competencies 
in the worlq>lra (Sticht, 1988; Mickulecky, 1988; PhiUipi, 1988,1990). 

Grey (19S6) in his UNESCO definitiim and Guthrie and Kiisch (1973) indicated that there 
may be a third level of literacy. They diiansed the nonism of an iiuiividual dev^oping 
Uteracy to a pdm where they could oontxid and improve their qiality of life. Perhaps this 
is tte baas for empowerment. It may not occur until an individual proceeds {ml the access 
and reqiured levels and into the i mpr ove me nt levd where an individual is able to set the 
paxam^as on what literacy skills are required. In mher '«ttings, it is the contoo, the work 
ewimmient, the task and otto people wi»s^ the requirements. Wten individuais have the 
skills to improve dieir own life, they are able to ^ tbo parameters themielves. 

The notions that levels of literacy exist means that it is appnqwiate to pursue Glasser's 
definidon of criterion scales (Itotd, 1984). That is we can define progressions of 
increasing proficiency in the kinds of litoacy. We do not however need to be confined to 
precise (tefinitions of the three levels above. A scale covning those arbitrary levels may be 
more useful if it were divided into a largn number of levels each indicating progr^. 

A Framework for literacy Scales 

A combination of the notion of kinds ami levels of literacy yidds a framework for the 
development of competency rating scales and profiles. 



FACET S L R W N D 



IXVEL 



Improve x x x x x x 



Require x x x x x x 



Access X X X X X X 



The Content of the Scales 

There are several methods available to identify the content of the scales. Cronbach (1978) 
offers an avenue of progress. He outlines a procedure which he calls the actuanai method. 
This involves finding individuals who already possess the characteristics which we seek to 
define and describe them. Groups of individuals become the yardsticks for the development 
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of the scale, at least in obtaining tlie criteria to place on the scales. This breaks us out of 
the dependeoct on testing and task analysis, and from the uneasy situation which links the 
competency to the task. Perhaps the most fruitful method of developing scales cf literacy 
is to combine all of these approaches. 

Tte combination can lead to tte ctesoiptiim of devefa^ent in each of the kinds of literacy. 
If we can analyse the tasks to ittentify the underpinning compiitencies, we may be able to 
identify the evidence of devdi^nnent in each kind of literacy. This was the case in the 
developnmnt of the literacy profiles (GrifBn, 1990, Victoria, 1990). 

The initial procedure is to identify and classify tasks that are consittered important in adult 
literacy. These may inducte the tasks of the NAEP te^ but will also include many other 
tasks ccmsidoed by teachers and those in the workplace to be important. An analyas of 
those tasks will identify underiying competencies and there may be sewral competrades 
associated with each task. This is an impcmant teasra to jooceed with the actuarial analysis 
rather than rely on the sole use the test based ap^oach. Testing might t» depli^yed to help 
in interpreting the levels of a scale developed this way. 



Alterative Approaches to Developing Scales 

An alternative would be to tate the approach adopted by the NAEP study of youth literacy 
and the Aumlian use of the test items for all adults. In that a^noach, set tasks were 
identified which were argued to represent the literacy tasks undertaken in every day life. 
Whetter all Australian adults per^mn the same tasks as Amoican youth is not 
addre ssed. The tasks are based on the KirsO* and Jungeblut defudtiim of e^ of the forms 
of literacy. The ddlnitions described spedfic task types which could be te^ such as 
computing the tip on a bill, completing a bank deposit slip or reading the label on a medicine 
bottle. Each of these tasks served to define functional literacy, which we discussed as bang 
related to the required level of literacy. 

If we return j) the notion of adult literacy and its orientation towards tasks, the difficulty 
that the underpinning competencies might not transfer from one task to anothn suggests that 
sole use of the test item approach may not give sufficient informaticn about literacy. Indeed 
we can define many tasks and test large numbers of individuals on those tasks. However if 
Mickelecky and othm are correct, this may lead to difficulties in generalising beyond the 
specific tasks tested. Indeed Wickert ( 1989) reported only the resulu on specific tasks. There 
was no evidence presented or claims made about the generaUsability of the results or even 
that the results could be developed into a scale of literacy. 



The Preferred Process 

The process adopted for this study begins with an analysis of the behaviours that individuals 
demonstrate in convincing teachers and others that these competencies nave been established. 
For each competency idoitified as underpinning a parucular task, we exar-me the kinds of 
tehaviour that iruiividuals exhibit - first that this competency is fully establLhed, secoiui that 
the competency may be just developing and third that the competency may be beginning or 
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even absent. This is illustiated in the figure below which reptesents a worksheet used for 
the pro c es s . 

ti t 

eaapctancv 1 eamtanef 2 ee^Mtaney S 
Cvldanea . - - 

Laval 

EttMlfsltad 
Davaloping 



M»am. 



uefkmh— t f o r IdtntlfvtfW eviawif of Cmwwwev Pwlecwwit. 

The evidence written in the cells of the table provides the indicatois of literacy. They indicate 
that a ccnMence is present, developing or absent This becomes the dau used in the 
developmoit of the competency ratings scales of adult litexacy. 

Iiuiicators of all of the comp^oicies may be used in surveys of adults, using expen judges 
to cover as wide a range of development as possible in each of the kinds of literacy. They 
are sorted using an item response theory analysis (Masters, 1982) to form a descriptive 
critniim scak. The indicators in the descriptive scales can be organised according to tteir 
relationship to one another. There are posably thousands of indicatcns and dncriptions of 
perscmal behavicHir that indicate different levds of literacy. Matrix sampling (Sirotnik, 1978) 
enables the use of a few indicators for each individual, and overlapping sets of indicaton to 
map all of diem onto a set of scales. 

What emerges for each scale is a kind of pyramid of indicators. At the bottom of the pyramid 
are behaviours that almost everyone exhibits. At the top of the pyiamid are bdiaviours that 
very few people exhibit. The relationship between the indicators is important. Indicators 
behave together to form a cohesi^ e set within each of the m^or components or kinds of 
literacy. They mm have an implicational relationship. That is an individual exhibiting 
behaviours at the top of the pyramid is likely to exhibit behaviours below that level. 
Exhibiting behavioun at any level does not imply that behaviours atove are present, but it 
does imply that behaviours below are likely to be present. Behaviours which do not fit 
within this implicational r^onship are excluded. This enables a cohesive s« of 
bdiaviours to be identified which will evennially provide a set of competency rating scales 
from which individual profiles can be developed. Hence the kinds and levels of literacy 
development can be described in terms of observable behaviours. 



Anchoring the Descriptive Scales 

Because the scales will ultimately be judgement based, it is necessary to check their qualities. 
Judgements on descriptive scales have been shown to be unreliable unless accompanied by 
some form of moderation. The ASLPR, a descriptive scale used to report adult migrant 
levels of language development, has a training package associated with it, and extensive use 
of moderation procedures help to maintain levels among experienced users. The same 
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difficulties will develop with the litoacy Competency Scales unless moderation procedures 
are devdoped, trialed aiul validated. Some form of external calibiatxim of raters judgements 
are also necessary. For numeracy and reading, this process is essentially simple. Various 
forms of standardiMd tests might be used to establish the pnq)erties of the scales. 

Exclusive rdiaii^ oa say single monitoring scteme is not a^nopriate. For example 
exclusive use of a test-based notion of literacy scal^ necessarily involves an assessor setting 
tasks for adults tr, peiform in order to establish a level of litoacy. This is particularly true 
of testing pK^xams, givm Aat te^ am oiUeciiims of tadcs, normally admini^ered by 
another poson. Testing renoins as an important as an wtemal validation of the scales as 
are e x per ien ces of individuals, judgements of experts, and otiier means of assessing the 
literacy devdf^meat against die pn^lte scales. Ifowever few mea»ires appear to be 
available across all of the scales. The NAEP item bank obviou^y ofka cme external check 
again» which to anchor the (tescr^nive scales. One odm possible external ai^or scale 
might be the Degrees of Heading Power (Bunill, 1987) a comprehension based test of 
reading based on dc^ te^g of extouted prose of materials suitable for adults. In &ct 
levels of performai^ have already been mappcfi onto an oonipsttional scale for a major 
insurance company in Jie United Stat» (BurriU, 1987). The r^xming and assessment model 
proposed by Griffin, (199(^), indicates that the scales are the basis of communication. The 
^tual asseiunent can take a varitty of forms, tests, work-ta^, direct observation, projects, 
including self assessment. 
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